Adjunctive ultrasound processing and display for breast cancer screening

ABSTRACT

An adjunctive ultrasound mammography system and associated methods are described comprising an adjunctive ultrasound display configured for quick, intuitive, interactive viewing of data derived from volumetric ultrasound scans, the data being displayed near a conventional x-ray mammogram display. Preferred embodiments for navigating among a thick-slice image array, a selected enlarged thick-slice image, and planar ultrasound views are described, including a preferred embodiment in which the planar ultrasound views are updated in real time as a cursor is moved across an active thick-slice image. In one preferred embodiment the thick-slice images are inverted prior to display, with non-breast areas of the image preferably segmented out and reset to dark. The inverted thick-slice images are of more familiar significance to radiologists having years of expertise in analyzing conventional x-ray mammograms. For example, the inverted thick-slice images allow benign features to be more easily dismissed as compared to non-inverted thick-slice images. Preferred embodiments for computing thick-slice image values from the volumetric scan data are also described that emphasize larger mass lesions in the resulting thick-slice images, and that compensate for mass lesions that straddle thick-slice region borders.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part of U.S. Ser. No. 10/160,836,filed May 31, 2002, which is a continuation-in-part of InternationalApplication Ser. No. PCT/US01/43237, filed Nov. 19, 2001, which claimsthe benefit of U.S. Provisional Application No. 60/252,946, filed Nov.24, 2000, each of these applications being incorporated by referenceherein. The above-mentioned Ser. No. 10/160,836 also claims the benefitof Provisional Application No. 60/326,715, filed Oct. 3, 2001, which isincorporated by reference herein. This application also claims thebenefit of Provisional Application No. 60/415,385, filed Oct. 1, 2002.The subject matter of the present application is related to the subjectmatter of Ser. No. 10/305,661, filed Nov. 27, 2002, which isincorporated by reference herein.

FIELD

This patent specification relates to medical imaging systems andprocesses. In particular, the present invention relates to theprocessing and display of breast ultrasound information in a manner thatefficiently and intuitively complements traditional x-raymammogram-based breast cancer screening methods.

BACKGROUND

Breast cancer is the most common cancer among women other than skincancer, and is the second leading cause of cancer death in women afterlung cancer. The American Cancer Society currently estimates that thereare about 203,500 new invasive cases of breast cancer per year amongwomen in the United States and 39,600 deaths per year from the disease.Prevention and early diagnosis of breast cancer are of foremostimportance. Because early breast cancer does not produce symptoms, theAmerican Cancer Society recommends a screening mammogram and a clinicalbreast examination every year for women over the age of 40.

X-ray mammography is currently the only imaging method for massscreening of breast cancer. In health maintenance organizations (HMOs)and other medical organizations, specialized x-ray mammography clinicsdesigned for high patient throughput are being increasingly used toscreen as many women as possible in a time and cost efficient manner.Numerous studies have shown that early detection saves lives andincreases treatment options. Recent declines in breast cancer mortalityrates (e.g., 39,600 deaths in 2002 versus 41,200 in 2000) have beenattributed, in large part, to the regular use of screening x-raymammography.

It has been found that the use of ultrasound mammography(sonomammography) in conjunction with conventional x-ray mammography candrastically increase the early breast cancer detection rate. Whereasx-ray mammograms only detect a summation of the x-ray opacity ofindividual slices over the entire breast, ultrasound can separatelydetect the acoustic impedance of individual slices of breast tissue, andtherefore may allow detection of breast lesions where x-ray mammographyalone fails.

However, as discussed in Ser. No. 10/160,836, supra, despite strongevidence that use of independent ultrasound examination would improveearly breast cancer detection and therefore save lives, substantialresistance against such use currently exists in the medical industry,including the radiologists themselves, and among policymakers. As usedherein, the term “radiologist” generically refers to a medicalprofessional that analyzes medical images and makes clinicaldeterminations therefrom, it being understood that such person might betitled differently, or might have differing qualifications, depending onthe country or locality of their particular medical environment. Severalinterrelated factors are often cited, including: (i) the false negative(missing) rate of independent ultrasound examination is unknown, (ii)the false positive rate of independent ultrasound examination is knownto be very high, leading to an increase in unneeded patient callbacksand biopsies, (iii) lack of image acquisition standardization, leadingto variability among different operators and radiologists, (iv) theadditional time and equipment required to conduct the ultrasoundexamination, leading to an increase in cost, (v) most if not allradiologists are not trained to read screening ultrasound images, whichcontain features not found in current breast imaging textbooks or taughtin current medical school courses, leading to a potential increase infalse negative (missing) rate and in the additional radiologist timerequired to analyze the ultrasound images, and (vi) the additionaltraining and clinical experience that would be required for theradiologist to properly analyze the ultrasound images.

Various schemes have been proposed for processing and presenting breastultrasound information in conjunction with x-ray mammogram informationfor use in breast cancer detection environments. In U.S. Pat. No.5,938,613, which is incorporated by reference herein, a method andapparatus for performing sonomammography and enhanced x-ray imaging isdiscussed in which ultrasound equipment is integrated with mammographyequipment to generate ultrasonic images of the breast that are ingeometric registration with an x-ray mammogram. An x-ray mammogram imageof an immobilized breast is acquired and, while the breast is stillimmobilized, an ultrasound scan is acquired using an automatedultrasound probe translation mechanism. Cross-sectional ultrasonicslices are summed across the entire breast to form a two-dimensionalultrasound image, which is then overlaid onto the digitized x-ray imagefor viewing by the radiologist. Precise geometric registration betweenthe ultrasound image and the x-ray mammogram is automatically providedbecause the breast is immobilized between imaging procedures and becausethe coordinates of the ultrasound probe are known during each scan. Theradiologist is permitted to instantiate certain algorithms such asdigital subtraction between the registered medical images.

However, the '613 patent is deficient in several respects with respectto the practical, real-world factors associated with the currentresistance against the use of ultrasound in mass breast cancer screeningenvironments. For example, the large base of currently installed x-rayimaging systems would require substantial retooling to accommodate themechanical apparatus of the '613 patent that keeps the breastimmobilized between imaging procedures and that performs the automatedultrasound scans. As another example, by displaying a summationultrasound image of all breast slices together, the '613 method deprivesthe radiologist of the ability to view individual planes inside thebreast. More generally, the computer-registered, static overlay of thesummation ultrasound image onto the x-ray image affords only a limitedamount of ultrasonic information to the radiologist as compared to theactual amount of ultrasonic data actually acquired, and affords onlylimited perception by the radiologist of structures within the breast.

In U.S. Pat. No. 5,662,109, a method and system for multi-dimensionalimaging and analysis for early detection of diseased tissue isdiscussed. Ultrasound scans of a breast are processed into multiplelayers of two-dimensional images, thus yielding a three-dimensional dataset. This data set and a two-dimensional x-ray mammogram are input to anenhancer that performs one or more “data fusion” algorithms to generatea three-dimensional representation of the breast for viewing. Theenhancer includes a registration module that expands and/or reducesdimensions of the data to register and align the ultrasound andmammographic images.

However, it is not believed that the various three-dimensional views ofthe “fused” data discussed in the '109 patent, such as the perspectiveview shown in FIG. 1 thereof, would be useful to a typical radiologisttrained in conventional x-ray mammography methods. As described supra,radiologists typically spend many years developing expertise inanalyzing a very particular set of two-dimensional x-ray mammographicdata taken from standardized views, most commonly the craniocaudal (CC)and mediolateral oblique (MLO) views. It is believed that mostradiologists would be reluctant to “start over again” with an entirelynew, different way of viewing the complex structures of a breast, andthat the medical industry would likewise be reluctant to forceradiologists to accept these viewing methods.

In view of the above discussions, it would be desirable to provide anadjunctive ultrasound mammography system that integrates ultrasoundmammography into current breast cancer screening methodologies.

It would be further desirable to provide an adjunctive ultrasoundmammography system that displays breast ultrasound information in amanner that facilitates the radiologist's perception of internal breaststructures that may not be readily apparent in an x-ray mammogram, whilealso being able to confirm the radiologist's perception of internalbreast structures that are apparent in the x-ray mammogram.

It would be even further desirable to provide an adjunctive ultrasoundmammography system that displays breast ultrasound information in amanner that supplements, rather than replaces, conventional x-raymammogram viewing methods, thereby increasing the likelihood of adoptionby both individual radiologists and the medical industry.

It would be even further desirable to provide an adjunctive ultrasoundmammography system that takes little or no special familiarization ortraining from the radiologist in order to effectively view breastultrasound information.

It would be still further desirable to provide an interactive userinterface for an adjunctive ultrasound mammography system that allowsthe radiologist to quickly and intuitively navigate among differentrepresentations of the breast ultrasound information.

It would be even further desirable to display such breast ultrasoundinformation in a manner that allows benign features to be more easilydismissed by the viewing radiologist.

SUMMARY

An adjunctive ultrasound mammography system and associated methods areprovided including an intuitive, interactive user interface fordisplaying breast ultrasound information to a user. According to apreferred embodiment, an array of thick-slice images derived fromvolumetric ultrasound scans of a breast is displayed, each thick-sliceimage representing a thick-slice or slab-like region of the breastvolume substantially parallel to a standard x-ray mammogram view of thebreast. Responsive to a first single-click or single-movement userselection of a first point on one of the thick-slice images, an enlargedview of that thick-slice image is displayed with a cursor positioned ata corresponding point. Responsive to a second single-movement userselection of a second point on the enlarged view, a first planarultrasound image encompassing the second point is displayed, the firstplanar ultrasound image representing the volumetric ultrasound scansalong a first plane substantially nonparallel to, and preferablyperpendicular to, the orientation of the slab-like region for thatthick-slice image.

According to another preferred embodiment, a second planar ultrasoundimage is shown concurrently with the first planar ultrasound imagerepresenting the volumetric scans along a second plane substantiallyorthogonal to both the first plane and to the orientation of theslab-like region. According to another preferred embodiment the firstand second planar ultrasound images are displayed concurrently with theenlarged thick-slice image or the array of thick-slice images. The firstand second planes correspond to the current cursor position on an activeone of the thick-slice images and are updated in real time as the cursoris moved. Range markers are provided on the planar ultrasound imagescorresponding to the current cursor position and to the borders of theslab-like region for the active thick-slice image.

According to another preferred embodiment, first and second planeindicators are displayed on the active thick-slice image, the planeindicators corresponding to the first and second planes and appearing asstraight lines for a default configuration in which the first and secondplanes are orthogonal to each other and to the orientation of theslab-like region for the active thick-slice image. In the defaultconfiguration, the first and second plane indicators intersect thecursor on the active thick-slice image. The user is permitted to departfrom the default configuration if desired by moving the first and secondplane indicators in a manner analogous to the way lines are moved in acomputer-aided drawing system, while the first and second planes and thefirst and second planar ultrasound images are updated in real time tocorrespond to the orientations and locations of the first and secondplane indicators.

A user interface according to the preferred embodiments is preferablyprovided in conjunction with an x-ray mammogram viewer such that thearray of thick-slice ultrasound images is displayed in coordination witha corresponding x-ray mammogram image taken from the same standard x-raymammogram view. The x-ray mammogram image, which is preferably providedon a backlighted film display but which can alternatively be provided onan electronic display, is displayed in close proximity to the array ofthick-slice ultrasound images to allow easy back-and-forth viewing.Preferably, the thick-slice images are displayed at full scale on an LCDmonitor positioned directly below the x-ray mammogram images, while thefirst and second planar ultrasound images are displayed on smaller CRTdisplays positioned to the sides of the LCD monitor. However, a varietyof different configurations having differing advantages are within thescope of the preferred embodiments as described further infra.

According to one preferred embodiment, the displayed thick-slice imagesare inverted to represent high acoustic reflections as “dark” and lowacoustic reflections as “bright,” in distinction to a standardultrasound display convention in which low acoustic reflections aredisplayed as “dark” and high acoustic reflections are displayed as“bright.” Preferably, the breast area is digitally segmented from thesurrounding area, and the surrounding area is reset to “dark” prior todisplay of the inverted thick-slice image. The inverted thick-sliceimages are of more familiar significance to radiologists having years ofexpertise in analyzing conventional x-ray mammograms. For example, theinverted thick-slice images allow benign features to be more easilydismissed as compared to non-inverted thick-slice images.

According to another preferred embodiment, a method for computing thethick-slice images from the volumetric ultrasound representation of thebreast is provided, each thick-slice image pixel being computed based onthe statistics of a voxel column passing through that location from alower border to an upper border of the relevant slab-like region. Inparticular, the statistical properties of interest are ones that incurchanges across different pixel locations in mass localities that aremore significant for masses greater than a preselected size of interestand that are less significant for smaller masses. Accordingly, masslesions greater than the preselected size of interest are emphasizedwhile smaller mass lesions are de-emphasized in the resultingthick-slice image. In one preferred embodiment, the thick-slice pixelvalue is selected as that value for which a cumulative distributionfunction (CDF) of the voxel column becomes equal to a threshold value,the threshold value being a predetermined fraction of a ratio of thepreselected size of interest to the distance between the first andsecond border planes. A method for ensuring the visibility of lesionsstraddling the borders between adjacent thick-slice regions is alsoprovided in which (i) an actual result for the actual thick-slice regionis computed, (ii) a hypothetical result is computed for a hypotheticalthick-slice region that is partially elevated into the adjacentthick-slice region, and (iii) resetting the actual result to thehypothetical result if the hypothetical result is more indicative oflesser ultrasound reflections.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a conceptual diagram of a system and method forbreast cancer screening using adjunctive ultrasound mammographyaccording to a preferred embodiment;

FIG. 2 illustrates steps for breast cancer screening using adjunctiveultrasound mammography according to a preferred embodiment;

FIG. 3 illustrates steps for interactively displaying adjunctiveultrasound mammography information to a user according to a preferredembodiment;

FIG. 4 illustrates an adjunct ultrasound display according to apreferred embodiment presenting an array of inverted thick-slice images;

FIG. 5 illustrates an adjunct ultrasound display according to apreferred embodiment presenting an enlarged inverted thick-slice image;

FIG. 6 illustrates an adjunct ultrasound display according to apreferred embodiment presenting a planar ultrasound image;

FIG. 7 illustrates an adjunct ultrasound display according to apreferred embodiment presenting an array of non-inverted thick-sliceimages for the same breast illustrated in FIG. 4;

FIG. 8 illustrates an adjunct ultrasound display according to apreferred embodiment presenting an array of inverted but non-segmentedthick-slice images for the same breast as FIGS. 4 and 7;

FIG. 9 illustrates an adjunct ultrasound display according to apreferred embodiment presenting an array of inverted thick-slice images;

FIG. 10 illustrates an adjunct ultrasound display according to apreferred embodiment presenting an enlarged inverted thick-slice image;

FIG. 11 illustrates an adjunct ultrasound display according to apreferred embodiment presenting a raw ultrasound image;

FIG. 12A illustrates a conceptual view of a thick-slice region andlesions contained therein, along with related histograms and cumulativedistribution functions;

FIG. 12B illustrates a conceptual view of a thick-slice region andlesions contained therein, along with plots of thick-slice image valuesalong a single line in the thick-slice image resulting from thick-sliceimage computation algorithms according to preferred embodiments;

FIG. 13 illustrates a conceptual frontal view of a set of neighboringthick-slice region and lesions contained therein, along with conceptualthick-slice images;

FIG. 14 illustrates steps for interactively displaying adjunctiveultrasound mammography information to a user according to a preferredembodiment;

FIG. 15 illustrates an adjunct ultrasound display according to apreferred embodiment presenting an array of inverted thick-slice imagesand two planar ultrasound images;

FIG. 16 illustrates the adjunct ultrasound display of FIG. 15 when auser moves the cursor on a selected thick-slice image;

FIG. 17 illustrates an adjunct ultrasound display according to apreferred embodiment comprising an enlarged inverted thick-slice imageand two planar ultrasound images;

FIG. 18 illustrates an exterior view of an adjunctive ultrasoundmammography display unit according to a preferred embodiment;

FIG. 19 illustrates a keypad for the adjunctive ultrasound mammographydisplay unit of FIG. 18; and

FIG. 20 illustrates a closer view of display portions of the adjunctiveultrasound mammography display unit of FIG. 18.

DETAILED DESCRIPTION

FIG. 1 illustrates a conceptual diagram of a system 100 and associatedmethods for breast cancer screening using adjunctive ultrasoundmammography according to a preferred embodiment. Adjunctive ultrasoundmammography refers to the acquisition and display of breast ultrasoundinformation during the breast cancer screening process in a manner thatsupplements x-ray mammogram information. System 100 comprises anultrasound scanning station 102, a computer network 104, an adjunctiveultrasound server 106, and an adjunctive ultrasound screening station108.

Ultrasound scanning station 102 comprises an apparatus designed toflatten and immobilize a breast while volumetric ultrasound scans areacquired. The breast is flattened along a plane substantially parallelto a standard x-ray mammogram view plane such as the CC and MLO viewplanes, although the ultrasound scanning station 102 is capable offlattening the breast along a variety of other planes as well.Ultrasound scanning station 102 comprises a housing 110 movablysupporting a gantry 114, the gantry 114 in turn supporting an uppercompression/scanning assembly 112 and a lower compression plate 113 in avertically movable manner.

For clarity of description herein, the y direction represents thehead-to-toe direction with respect to the patient, the x-axis representsthe left-to-right direction, and the z direction extends outward fromthe chest wall. The x-y, y-z, and x-z planes thus correspond to thecoronal, sagittal, and axial planes, respectively. The patient may standor sit in front of the ultrasound scanning station 102, facing the +zdirection in FIG. 1, with one breast placed between the uppercompression/scanning assembly 112 and lower compression plate 113.Responsive to control by an operator using foot pedals 115, a keyboard118, a mouse 117, buttons on the gantry 114, and/or other input methods,the breast is compressed and an ultrasound probe head (not shown)contained inside upper compression/scanning assembly 112 is linearlytranslated over the top of the breast while two-dimensional ultrasoundslices are acquired. Ultrasound scanning station 102 further comprisesan ultrasound processor (not shown) coupled to the ultrasound probe headthat receives the acoustic echo signals and forms the two-dimensionalultrasound slices therefrom. The ultrasound slices may be viewed inreal-time on a monitor 116 as they are generated, the monitor 116 alsoserving as an interface display for controlling of the overall operationof the ultrasound scanning station 102.

Preferably, the upper compression/scanning assembly 112 is similar tothat described in Ser. No. 60/415,385, supra. The breast skin surfacecontacts one side of a taut sheet of acoustically transparent materialsuch as Mylar® while the other side of the taut sheet is in actual orimminent contact with the probe head. Acoustic coupling between the tautsheet and the probe head is facilitated by a stream, drip, or bath ofwater or other low-viscosity, acoustically conductive fluid. The gantry114 is rotatable in a plane parallel to the coronal plane of thepatient, i.e., around the z-axis in FIG. 1, such that scans of thebreast flattened along the MLO plane or other view planes can beachieved. The gantry 114 can also be tilted forward or backward relativeto the patient, i.e., around the x-axis in FIG. 1, between a range ofroughly plus 30 degrees to minus 30 degrees.

According to one preferred embodiment, a breast scan for a given view isacquired by a single sweep of the probe contained within uppercompression/scanning assembly 112. In this case, it is required that thescan penetration depth extend as far as possible toward the lowercompression plate. For larger breasts, this can be 6 cm or greater, inwhich case a lower probe frequency is required and a correspondinglylesser resolution is obtained than for high-frequency scans. Accordingto another preferred embodiment, dual sweeps can be taken for a givenview, with the gantry being rotated 180 degrees around the y-axis inFIG. 1 between sweeps. In this case, the scans only need to penetratethrough half the breast thickness and so a higher scan frequency can beused. The resulting ultrasound “half-slices” from the first and secondsweeps can then be stitched together to form complete slices. In stillanother preferred embodiment, the lower compression plate 113 isreplaced with a second compression/scanning assembly including a secondprobe head. In this case the separate upper and lower probe sweeps canbe achieved without requiring the intermediate 180 degree rotation ofthe gantry, and issues relating to registration between the half-slicesare avoided.

During or after the ultrasound scanning process, the raw ultrasound datais provided across the computer network 104 to the adjunctive ultrasoundserver 106, where the raw ultrasound data is processed into adjunctiveultrasound data that will be made available to the screeningradiologist, the adjunctive ultrasound data including ultrasound slices,thick-slice images, CAD outputs, and other useful information. It is tobe appreciated that the processing of the raw ultrasound data into theadjunctive ultrasound data may be performed by any of a variety ofdifferent computing devices coupled to the computer network 104 and thentransferred to the adjunctive ultrasound server 106.

In current mass breast cancer screening environments based on x-raymammography, a screening radiologist 124 examines x-ray mammograms formany patients en masse in a single session using an x-ray viewingstation 109. The x-ray viewing station 109 may range from a simple lightbox, as in FIG. 1, to more complex x-ray CAD workstations thatautomatically move the x-ray mammograms past the radiologist 124 on aconveyor belt as a nearby CAD display highlights suspicious areas of themammogram. Almost universally, left and right CC x-ray views 120 arepositioned on one side of the x-ray viewing station 109, and left andright MLO x-ray views 122 are positioned on the other side. Theradiologist 124 quickly examines the x-ray mammograms. For some x-raymammograms the radiologist needs only a few seconds, while for otherx-ray mammograms the radiologist needs up to five minutes, with anaverage being about two minutes per mammogram.

According to a preferred embodiment, this existing arrangement remainssubstantially undisturbed, but is augmented with equipment and data thatfacilitates fast and thorough x-ray mammogram screening by giving theradiologist a quick ultrasonic “second look” at the internal breaststructure. Adjunctive ultrasound screening station 108 comprises firstand second adjunct displays 126 and 128 conveniently positioned near thex-ray viewing station 109 such that the radiologist 124 can (i) easilylook back and forth between the first adjunct display 126 and the CCx-ray views 120, and (ii) easily look back and forth between the secondadjunct display 128 and the MLO x-ray views 122. Preferably, adjunctdisplays 126 and 128 display thick-slice images 136 and 138,respectively, corresponding to thick-slice regions of the breast volumesubstantially parallel to the CC and MLO view planes, respectively,acquired while the breast was flattened along the CC view plane and theMLO view plane, respectively. This allows the spatial content of thethick-slice images to roughly correspond to the spatial content of thecorresponding x-ray mammograms, facilitating ready comparisonstherebetween. However, the scope of the preferred embodiments is notnecessarily so limited. According to an alternative preferredembodiment, the benefits of meaningful “second look” information insidethe breast structure is still provided even where (i) the breast iscompressed along a non-standard plane during the volumetric scans, or(ii) the breast is not compressed at all during the volumetric scans, or(iii) the thick-slice images correspond to planes not parallel to astandard x-ray mammogram view plane. In view of the thick-slicesegmentation and inversion process described herein and other featuresand advantages according to the preferred embodiments, such non-standardcompressions or non-standard thick-slice planes can still result usefulthick-slice adjunctive ultrasound images for viewing, especially where aspatial guide similar to the iconic representations infra are displayedto properly “orient” the reader to the position and orientation of thenon-standard thick-slice image.

According to a preferred embodiment, adjunct displays 126 and 128 aredesigned to facilitate quick, intuitive, and interactive navigationamong different views of the thick-slice images and other adjunctiveultrasound data. Adjunct displays 126 and 128 are preferablytouch-screen displays but other input devices just as a PC keyboard andmouse (not shown) can be used. FIG. 1 also shows control buttons 134 and140 on adjunct displays 126 and 128, respectively, that have layouts andfunctionalities described further infra. A bar code reader 143 reads abar code of the x-ray mammogram, wherein the associated adjunctiveultrasound data for that breast is automatically retrieved from theultrasound server 106. In the event that the x-ray mammograms are loadedonto a motorized viewer, the bar codes from the x-ray mammograms areread as the images are loaded into the apparatus by a technician.Alternatively, the user can scan the bar code directly from the x-raymammogram when it appears in front of them using a hand-held bar codescanner, wherein the corresponding adjunctive ultrasound data isretrieved from the adjunctive ultrasound server 106. For clarity ofpresentation, the user interface description herein is presentedrelative to the CC adjunct display 126, it being understood thatanalogous descriptions apply to the MLO adjunct display 128 or tonon-standard-plane adjunct displays in general. Additionally, althoughmost of the exemplary displays herein show the CC data for a singlebreast (left or right), it is to be appreciated that concurrent viewingof the CC data for the other breast on the same display, or combinationsof CC/MLO/non-standard views for both left and right breasts, areclearly within the scope of the preferred embodiments.

The adjunct display 126 of FIG. 1 contains an array of smaller, orthumbnail, thick-slices images 136, with the terms small and thumbnailsimply indicating that the images are small enough to fit on the samemonitor while communicating structures of more than one thick-sliceregion of the breast, and are smaller in size than the enlargedversions. Due to practical display size limitations, it is expected thatthe small or thumbnail images will be less than full-scale images andthe enlarged versions will be greater than full-scale, although thescope of the preferred embodiments is not so limited. In one alternativepreferred embodiment both the thumbnail and enlarged versions are lessthan full-scale, while in another preferred embodiment the thumbnailimages are at full-scale and the enlarged versions are greater thanfull-scale, while in still another alternative preferred embodiment thethumbnail and enlarged versions are both greater than full-scale.

As described in Ser. No. 10/160,836, supra, the thickness of theslab-like or thick-slice volume corresponding to each thick-slice imagemay lie, for example, in the range of 2 mm to 20 mm, although the scopeof the preferred embodiments is not so limited, and thicknesses in therange of 7 mm to 12 mm are likely to be suitable for most breast cancerscreening purposes. Techniques for integrating the component ultrasoundslices into thick-slice images according to the preferred embodimentsinclude arithmetic averaging, geometric averaging, reciprocal averaging,exponential averaging, and other averaging methods, in each caseincluding both weighted and unweighted averaging techniques. Othersuitable integration methods may be based on statistical properties ofthe population of component ultrasound slices at common locations, suchas maximum value, minimum value, mean, variance, or other statisticalalgorithms. One particularly suitable algorithm for generatingthick-slice images from a volumetric representation of a breast isdescribed infra with respect to FIGS. 12-13.

FIG. 2 illustrates steps for breast cancer screening using adjunctiveultrasound mammography according to a preferred embodiment. At step 202an x-ray mammogram is obtained and at step 204 volumetric ultrasoundscan data is obtained. At step 206 the ultrasound scan data isassociated with the x-ray mammogram data for the patient, breast, view,date, etc., as generally described in parent application Ser. No.10/160,836, supra. At step 208 thick-slice images are formed from thevolumetric ultrasound scan data according to the method described infrawith respect to FIGS. 12-13. For CC thick-slice images, i.e.,thick-slice images representing thick-slice volumes substantiallyparallel to the CC plane, each thick-slice image is a scalar function ofcoordinates (x,z). At step 210 an optional step is performed whereincomputer-aided diagnosis (CAD) algorithms are applied to the ultrasoundscan data and/or x-ray mammogram data. In accordance with a preferredembodiment, the resulting CAD markers may be superimposed upon one ormore of the thick-slice images described throughout this application, onany of planar ultrasound views, on any of the x-ray mammogram views, orany combination of them to achieve a CAD-enabled display.

At step 212 the breast tissue is segmented from outlying areas using anyof a variety of known segmentation algorithms. Preferably, athree-dimensional segmentation algorithm is performed directly on thethree-dimensional volumetric scan data, although in other preferredembodiments a two-dimensional segmentation algorithm is separatelyapplied to each thick-slice image. The segmentation step 212 results ina mask for each thick-slice image identifying the breast tissueboundary.

At step 214 inverted thick-slice images are computed from the ordinaryor non-inverted thick-slice images. As used herein, a non-inverted orordinary thick-slice image generally conforms to a standard medicalultrasound display convention in which readings of lesser acousticreflections are displayed as “darker” (blacker, dimmer, darker gray,lower-intensity, etc.) and in which readings of higher acousticreflections are displayed as “brighter” (whiter, lighter,higher-intensity, etc.). An inversion algorithm converts eachnon-inverted thick-slice image pixel P₀(x,z) into a complementary orinverted value P_(INV)(x,z) that is then “brighter” in regions of lesseracoustic reflection and “darker” in regions of higher acousticreflection.

It has been found that displaying inverted thick-slice images cansubstantially enhance the viewing and screening process, and canfacilitate the dismissal of benign lesions better than a display ofnon-inverted thick-slice images. This is at least because, for invertedthick-slice images, differential viewing of breast lesions versussurrounding tissue structures is provided on a similar basis as that forx-ray mammograms. For example, most radiologists have developed years ofexpertise in differentiating “bright” lesions from surrounding ligamentson x-ray mammograms, the surrounding ligaments also being “bright” buthaving different visual cues. The use of inverted thick-slice imagesallows their years of expertise to be extended over to the thick-sliceultrasound data, in contradistinction to the conventional ultrasounddisplay method that would require the difficult task of differentiating“dark” lesions based on different visual cues than “dark” surroundingligaments.

Although any of a variety of inversion algorithms could be used inaccordance with the preferred embodiments at step 214, it has been foundbeneficial to use an inversion algorithm that also performs some degreeof contrast-enhancement when mapping the darker values of P₀(x,z) intothe brighter values of P_(INV)(X,Z). For an exemplary situation in whichthe display monitor pixels are brightest at value 255 and lowest atvalue 0, one particularly suitable algorithm is given in Eq. (1) below,with y (“gamma”) being set to 0.5:

$\begin{matrix}{{P_{INV}\left( {x,z} \right)} = {255\left( \frac{255 - {P_{0}\left( {x,z} \right)}}{255} \right)^{\frac{1}{\gamma}}}} & \left\{ 1 \right\}\end{matrix}$

During image inversion, non-breast areas of the thick-slice images,which are initially dark, are converted to bright as displayed in theexample of FIG. 8, infra. According to a preferred embodiment, thenon-breast areas of the thick-slice images, as identified by the maskspreviously computed at step 212, are reset to dark. This step isperformed so that the displayed thick-slice images are more reminiscentof an x-ray mammogram, which is dark in the unexposed regions lyingoutside the breast. It is to be appreciated, however, that this step canbe skipped without departing from the scope of the preferredembodiments, as some users may end up preferring the white background.

Generally speaking, the ultrasound processing steps 208-214 are usuallynot performed in real-time, but rather are performed during an intervalbetween the scanning process and viewing process, which can be a periodof several hours or more. However, the scope of the preferredembodiments is not so limited, and the steps 202-214 may also beperformed in real-time if practicable in a given clinical setting.

At step 216 the x-ray mammogram information is retrieved and displayedto the user. At step 218 the corresponding adjunctive ultrasound data isretrieved, including the inverted thick-slice images. At step 220 theuser views and analyzes the x-ray mammogram image. In a manner analogousto FDA-approved practices for “second-look” x-ray mammogram CAD results,the user should first examine the x-ray mammogram without reference tothe thick-slice images, first arriving at an independent conclusionbased on the x-ray mammograms alone. Only after the independent x-raymammogram analysis should the user view the thick-slice images (step222), wherein the user interacts with the adjunct ultrasound display asnecessary to confirm and/or supplement the x-ray mammogram analysis.Optionally, to ensure the proper order of viewing, the thick-sliceimages are withheld from view until the user verifies that anindependent x-ray mammogram analysis is completed by pressing, forexample, a confirmation toggle button or entering an appropriate usercommand.

FIG. 3 illustrates steps for interactively displaying adjunctiveultrasound mammography information to a user according to a preferredembodiment. FIGS. 4-6 illustrate an adjunct display screen 400 atparticular stages during the steps of FIG. 3. At step 302 an array ofinverted thick-slice images is displayed. FIG. 4 illustrates an exampleof a thick-slice image array 402 on the adjust display screen 400.Adjunct display screen 400 further comprises a patient ID section 404comprising patient identification and other relevant information.Adjunct display screen 400 further comprises a control button array 406comprising buttons that can be actuated by a touchscreen press or amouse/trackball inputs. Also shown in FIG. 4 is a selection marker 407appearing at a location 408 superimposed on a thick-slice image 410. Ifa mouse/trackball input is used, the default mouse pointer changes tothe selection marker 407 when guided over any one of the thick-sliceimages 402.

At step 304 a single-click or single-motion input from the user isreceived indicating a first selected location on a selected invertedthick-slice image. With reference to FIG. 4 this is achieved by making amouse click with the selection marker 407 at the current position (408)or by pressing the touchscreen at position 408. In the case of atouchscreen, it is recommended that a pointing device finer than a humanfinger be used, such as the stylus of a personal digital assistant(PDA), to touch the screen at the location 408. Responsive to thesingle-click or single-motion input, at step 306 an enlarged view 502 ofthe selected inverted thick-slice image 410 is displayed. A selectionmarker 503 is displayed at a location 504 that corresponds to the samerelative position on the breast as the location 408 on the thick-sliceimage 410.

Also shown in FIG. 5 is a three-dimensional icon 506 that roughlyillustrates the thickness, location, and orientation of the thick-slicevolume corresponding to the thick-slice image 502. In particular, theiconic thick-slice volume 508 is shown relative to an iconic rectangularsolid 510 that roughly corresponds to the compressed breast. Althoughnot illustrated as such in FIG. 5, it is preferable to display an iconicframe that more closely resembles a perspective view of the breastrather than the iconic rectangular solid 510. The iconic frame cancomprise a three-dimensional rendering of a semi-transparent breast, andthe iconic thick-slice volume 508 can be suspended at the appropriatelocation and orientation therein. Because the three-dimensional icon 506does not provide any specific diagnostic information to be relied upon,it is not necessary for the iconic frame to be a perfect replica of theactual scanned breast. However, to increase the intuitiveness of thedisplay, especially to new users, it is preferable that thesemi-transparent rendering be of at least sufficient quality to berecognizable as a breast volume.

At step 308 a second single-click or single-motion input is receivedfrom the user indicating a second selected location on the enlargedinverted thick-slice image. By default, the second selected location isat the same point relative to the enlarged inverted thick-slice breastas the first selected location was to the thumbnail or smallerthick-slice image, unless the user affirmatively moves the selectionmarker 503 prior to clicking. In the example of FIGS. 4-6, the secondselected location is at location 504 of FIG. 5. At step 310 a rawultrasound image or reconstructed thin-slice image encompassing thesecond selected location is displayed.

FIG. 6 illustrates a raw ultrasound slice 602 displayed responsive tothe second single-click or single-motion input. The raw ultrasound slice602 is preferably shown in a standard, non-inverted format because thisdirect form of ultrasound viewing is already familiar to most users.However, it is within the scope of the preferred embodiments for the rawultrasound slice 602 to be inverted prior to display. The raw ultrasoundslice 602 represents one example of a planar ultrasound imagerepresenting sonographic properties of the breast along a plane cuttingthrough the breast. In particular, the raw ultrasound slice 602represents a first planar ultrasound image representing values along afirst plane perpendicular to the orientation of the thick-slice regioncorresponding to the thick-slice image being displayed, wherein thefirst plane is also parallel to an ultrasound scan plane of theultrasound probe that is swept across the breast during the volumetricscanning process. Most examples of planar ultrasound images describedherein are raw ultrasound slices or orthogonal thin-slice images, theorthogonal thin-slice images being planar ultrasound images representingvalues along a second plane perpendicular to both (i) the raw ultrasoundimage, and (ii) the orientation of the thick-slice region correspondingto the thick-slice image being displayed. The orthogonal thin-sliceimages need to be reconstructed from the three-dimensional volumetricrepresentation, as opposed to the raw ultrasound slices which can betaken directly from the raw data used to construct the three-dimensionalvolumetric representation. It is to be appreciated that while mostexamples of planar ultrasound images described herein are raw ultrasoundslices or orthogonal thin-slice images, the scope of the preferredembodiments is not limited as such and extends to general cases of firstand second planar ultrasound images having differing orientations.

Several features are provided on the adjunct display of FIG. 6 to assistthe user in visualizing the location and orientation of the rawultrasound slice 602 relative to the breast volume. First, analphanumeric representation 603 of the actual raw slice number (in thisexample, slice number 300) out of the total number of raw slices (inthis example, 533 slices) taken during the scanning process is shown.The three-dimensional icon 506 including the iconic rectangular solid510 that roughly represents the breast volume outline is maintained inthis view, with the position and orientation of the raw ultrasound slice602 being shown therein by an iconic plane 604. In accordance with step312 of FIG. 3, upper and lower range markers 606 and 608, respectively,indicate the upper and lower “y” for the thick-slice image 502previously being viewed at a horizontal “z” position corresponding tothe selected location 504 from FIG. 5. Depth range markers 612 indicatethe vertical “y” extent for the thick-slice image 502 as well. Hashmarkers 610 also assist in locating the range markers 606 and 608.Finally, the location of range markers 606 and 608 relative to the rawultrasound plane is also indicated by an iconic marker 605 on thethree-dimensional icon 506.

In the example of FIG. 6, it is a raw ultrasound slice that is displayedbecause the ultrasound scans were taken directly in the y-z plane by theultrasound scanning device 102 of FIG. 1. In a more general case, theraw ultrasound scans may not have been taken in planes perpendicular tothe thick-slice region of the thick-slice image of FIG. 5. In thosecircumstances, a planar or thin-slice ultrasound image is reconstructedfrom the volumetric ultrasound data, the planar ultrasound imagerepresenting values along a plane perpendicular to the orientation ofthe relevant thick-slice region and passing through the second selectedlocation of FIG. 5. The planar or thin-slice ultrasound image ispreferably computed to represent as thin a slice as possible whileremaining accurate, or to represent a slice having a thicknesscorresponding to the elevation beamwidth of the linear probe used toscan the breast if such type of probe was used.

At step 314 cine viewing of raw ultrasound or thin-slice image frames isaccommodated, beginning at the frame of raw ultrasound slice 602. In thecase of a mouse input, the cine can be controlled by a mouse wheel ormouse-mounted trackball. In the case of a touchscreen display, cinecontrol can be achieved by a separate trackball provided near thedisplay. A foot pedal can also be used for cine control to free up thehands of the user. Onscreen cine control buttons 614 can also be used.

At step 316 single-click or single-motion return to the selectedenlarged inverted thick-slice image or to the array of invertedthick-slice images is accommodated. For example, a left-click of a mouse(or single-tap of a stylus on a touchscreen display) at a third selectedlocation on the raw ultrasound slice 602 can bring up the enlargedthick-slice image corresponding to “y” depth of the third selectedlocation, with a marker thereon indicating the (x,z) location of thethird selected point. Alternatively, an onscreen “one slab” onscreenbutton 616 can be pressed to bring up the previous enlarged thick-sliceimage 502. A right click of a mouse (or double tap of a stylus on thetouchscreen display) at the third selected point can bring up the array402 of thick-slice images, with markers indicating the (x,z) position ofthe third selected location superimposed on the appropriate one of thethick-slice images for the “y” depth of the third selected location.Alternatively, an “all slab” onscreen button can be pressed to bring upthe array 402 of thick-slice images.

FIG. 7 illustrates the adjunct ultrasound display screen 400 accordingto an alternative preferred embodiment in which an array 702 ofnon-inverted thick-slice images are displayed. The non-invertedthick-slice image array 702 corresponds to the same breast andultrasound scanning session as for the inverted thick-slice image arrayof FIG. 4. With reference to a particular region 704 circled in FIG. 7,it has been found that the non-inverted thick-slice images 702 have atendency to contain substantial areas of darkness in the breast tissue,wherein it is more difficult to detect subtle texture differences inbreast tissue because it requires the differentiation of “dark” lesionsfrom “less dark” or “differently dark” surrounding ligaments. This canlead to a larger number of false positives and also to user fatigue andfrustration. In distinction, the inverted thick-slice images of FIG. 4are easier to analyze, at least because of (i) the many years oftraining that most radiologists have in differentiating “bright” lesionsfrom “bright” surrounding ligaments in x-ray mammograms, and (ii) thecontrast enhancement performed at brighter output levels during theinversion step 214 supra.

FIG. 8 illustrates the adjunct ultrasound display 400 according toanother preferred embodiment presenting an array 802 of thick-sliceimages that were inverted but for which the background segmentation anddarkening steps were omitted. The thick-slice image array 802corresponds to the same breast and ultrasound scanning session as forthe thick-slice image arrays of FIG. 4 and FIG. 7. As indicated in FIG.8, the outlying non-breast areas are bright instead of dark. In additionto being less familiar to most radiologists trained on x-ray mammograms,it is believed that this preferred embodiment might also lead to greatereyestrain. However, this preferred embodiment might neverthelessrepresent a desirable display format for some nonzero population ofusers.

FIGS. 9-11 illustrate a thick-slice image array 902, an enlargedthick-slice image 1002, and a raw ultrasound slice 1102, respectively,in a manner analogous to the images 402, 502, and 602 of FIGS. 4-6, butfor a different patient. FIGS. 9-11 illustrate one of the many featuresand advantages of the preferred embodiments wherein in which lesionshadows among successive thick-slice images, or the lack thereof, canrapidly assist in a diagnosis. Circled in FIG. 9 are locations L-3, L-4,and L-5 to indicate a particular (x,z) location L at thick-slice zones3, 4, and 5, respectively. Notably, there is a white spot in the thirdzone at L-3, as illustrated on both FIG. 9 and FIG. 10, but there aredark spots in corresponding areas L-4 and L-5 of the fourth and fifthzones lying underneath the third zone. This indicates that there werelow acoustic reflection readings for the third zone at L-3, while therewere high acoustic reflection readings for two zones directly therebelowat L-4 and L-5. Accordingly, it is highly likely that this is a benignliquid-filled cyst that can be readily ignored. This can be quicklyverified by viewing the raw ultrasound slice 1102 of FIG. 11, whichshows a dark area 1104 of low reflections over an area 1106 ofaccentuated reflections.

Thus, according to a preferred embodiment, a method for quicklydiagnosing the presence of a benign, liquid-filled cyst is provided,comprising the step of viewing an array of inverted thick-sliceultrasound images including a first inverted thick-slice image and aplurality of neighboring inverted thick-slice images, the neighboringinverted thick-slice images corresponding to thick-slice regionsdirectly below the thick-slice region of the first inverted thick-sliceimage relative to an ultrasound detector. The method further comprisesthe steps of observing a bright spot in the first thick-slice image andsearching for dark spots in the plurality of neighboring invertedthick-slice images at locations corresponding to the bright spot. Ifsuch dark spots are present, the bright spot of the first invertedthick-slice image is diagnosed to be a benign, liquid-filled cyst. Themethod further comprises the step of verifying such diagnosis by viewinga raw ultrasound slice or reconstructed thin-slice image having a planethat passes through the breast location indicated by the bright.

With reference to FIG. 11, selected onscreen control buttons 406 are nowdescribed. An onscreen “select study” onscreen button 1150 activates afile selection dialog box in which the user can select adjunctiveultrasound data for a particular patient and scan session. Onscreen“view selection” buttons 1152 select the desired view of LCC, RCC, LMLO,and RMLO. Onscreen display selection buttons 1154 allow the user toselect the “all slab” view of FIG. 4, the “one slab” view of FIG. 5, the“standard” view of FIG. 6, or a “top” view representing an integrationof all thick-slice regions for that view. Onscreen “film box” buttons1156 allow for easy collection and/or printing of interesting images forsubsequent review.

FIG. 12A illustrates a conceptual view of a thick-slice or slab-likeregion 1202 substantially parallel to the CC plane, along withhypothetical mass lesions 1206, 1208, and 1210, presented for describingthick-slice image computation according to a preferred embodiment. Asindicated by the provided reference axes, the thick-slice region 1202 isbeing viewed from the +z direction, i.e., from the front of the woman.The thick-slice region 1202 has a thickness T which can be about 1 cmfor this example. The described thick-slice image computation algorithmsoperate on standard, non-inverted ultrasound voxel values for which zerois “dark” and 255 is “bright,” although it is to be appreciated that itcould be readily adapted by a person skilled in the art in view of theinstant disclosure to operate on inverted ultrasound voxel values orotherwise-remapped voxel values.

Shown in FIG. 12A are side views of pixel columns 1204, including pixelcolumns 1212, 1214, and 1216 passing through the hypothetical masslesions 1206, 1208, and 1210, respectively. A given pixel columnextending downward in the y direction passing through the thick-sliceregion at pixel location (x,z) could comprise, for example, 40-60voxels, each having an 8-bit value V_(xz)(y_(n)). For each pixel columnpassing through the thick-slice region 1202, a scalar output valueP₀(x,z) is computed that constitutes the thick-slice image at that pixellocation. In a simplest preferred embodiment, the thick-slicecomputation algorithm simply computes an average pixel value, asdescribed by Eq. (2) below, where N is the number of voxels in the pixelcolumn at (x,z):

$\begin{matrix}{{P_{0}\left( {x,z} \right)} = \frac{\sum\limits_{n = 1}^{N}\;{V_{xz}\left( y_{n} \right)}}{N}} & \left\{ 2 \right\}\end{matrix}$

However, it has been found that thick-slice images more useful in breastcancer screening can be achieved by using a thick-slice computationalgorithm that detects particular statistical variations ofV_(xz)(y_(n)) in the neighborhood of masses in a manner that emphasizesmasses larger than a predetermined target size and de-emphasizes smallermasses in the in the resulting thick-slice image.

Shown in FIG. 12A for each of the pixel columns 1206, 1208, and 1210passing through the respective hypothetical mass lesions is a histogramof the number of pixel elements at each respective gray value. Under thenon-inverted convention of FIG. 12, the mass lesions of interest forbreast cancer screening purposes tend to show up as “dark” regionsagainst “whiter” backgrounds. It is to be observed that pixel columnspassing through smaller masses, such as pixel column 1212 passingthrough mass 1206 having size S1, have substantially more high pixelvalues than low pixel values, as revealed by the associated histogramcurve. As such, the area under the histogram from zero to a pixel value,termed herein the CDF or cumulative distribution function ofV_(xz)(y_(n)), remains low for most pixel values and only increases ashigher pixel values are reached. Conversely, pixel columns passingthrough larger masses, such as pixel column 1214 passing through mass1208 of size S2, have substantially more low pixel values than highpixel values, as revealed by the associated histogram curve and CDF.

According to a preferred embodiment, a target size D is establishedrepresenting a mass size expected to be interesting from a breast cancerscreening point of view. For purposes of this example, and not by way oflimitation, a suitable target size D can be about 0.5 cm, which yields atarget size to slab thickness ratio (D/T) equal to 0.5. The output pixelvalue P₀(x,z) is set equal to that pixel value for which the cumulativedistribution function of V_(xz)(y_(n)) is equal to a preselected value Ktimes the target size to slab thickness ratio. This is graphicallyillustrated in FIG. 12 and expressed in Eq. (3) below:

$\begin{matrix}{{{CDF}_{V_{xz}}\left\lbrack {P_{0}\left( {x,z} \right)} \right\rbrack} = {K\left( \frac{D}{T} \right)}} & \left\{ 3 \right\}\end{matrix}$

FIG. 12B illustrates a conceptual view of the thick-slice region 1202and lesions 1252 contained therein for a single fixed value Ofz=z_(fixed), along with plots 1254 and 1256 of thick-slice image valuesP0(x,z_(fixed)) along a single line, the thick-slice images beingcomputed according to Eqs. (2) and (3), respectively. For ease ofcomputation and presentation, the lesions and surrounding breast tissueare assigned the extreme brightness values of 0 and 255, respectively.Although this conceptualizes the example somewhat by using an“extremely” bimodal voxel value distribution, it is to be appreciatedthat regions near mass lesions often have histogram distributions with aroughly bimodal characteristics, and so the results of FIG. 12B areroughly representative of the actual results obtained. A preselectedvalue of K=0.5 was used in the application of Eq. (3). As indicated by acomparison of plot 1254 versus the plot 1256, the use of the CDF-basedmethod of Eq. (3) yields a resulting thick-slice image that enhances thevisibility of lesions as they approach and exceed the target size D,while de-emphasizing smaller lesions. Different values of K may be usedin Eq. (3), e.g., in the range of 0.25 to 0.75, and the value may beoptimized for a given configuration based on empirical results.Generally speaking, as K is increased there is decreased “sensitivity”with more of the smaller lesions being ignored. In an alternativepreferred embodiment, the value of K may be dynamically assigned basedon local volume characteristics, for example, as a function of the localstatistical variance of the voxel values.

FIG. 13 illustrates a border compensation method for thick-slicegeneration according to the preferred embodiments that compensates forthe possibility that lesions may straddle thick-slice borders. Shown inFIG. 13 is an exemplary thick-slice region 1302 vertically adjacent toan upper thick-slice region 1306 and a lower thick-slice region 1304.Also shown in FIG. 13 are exemplary mass lesions A-G, some of which areentirely contained in one of the thick-slice regions and some of whichstraddle the thick-slice region borders. As indicated in FIG. 13, theterms upper and lower are indicative of a distance from an ultrasoundprobe used to scan the breast volume. According to a preferredembodiment, when computing a thick-slice image, two alternativethick-slice values for each location (x,y) are computed, including afirst value corresponding to the actual borders of the thick-sliceregion and a hypothetical value corresponding to the borders of ahypothetical thick-slice region having a similar thickness but extendinginto the next upper thick-slice region. In the example of FIG. 13, ahypothetical thick-slice regions 1302 a and 1304 a correspond tothick-slice regions 1302 and 1304, respectively. For each location (x,z)in the thick-slice image, the lesser (i.e., “darker”) of the first valueand the hypothetical value is used.

Also shown in FIG. 13 are conceptual thick-slice images 1302′, 1304′,and 1306′ corresponding to thick-slice regions 1302, 1304, and 1306,respectively. In this example, it is presumed that there are no lesionsin the breast other than the mass lesions A-G. For purposes ofpresentation, a filled-in circle is shown when the “entire effect” of amass lesion is present in the thick-slice image, whereas an empty circleis shown when only a “half effect” is present. Notably, whereas thelesion B would only be “half-present” in both of the thick-slice images1306′ and 1302′ in an uncompensated scenario (not shown), it is“fully-present” in the thick-slice image 1302′ as well as “half present”in thick-slice image 1306′ in the compensated scenario shown in FIG. 13.Lesion A, which would be fully-present in thick-slice image 1306′ andnot present in thick-slice image 1302′ in an uncompensated scenario, isfully-present in both of the thick-slice images 1306′ and 1302′ in thecompensated scenario. It has been found that this border-straddlingcompensation strategy, for which other examples are presented in FIG.13, represents a desirable result even though it tends to “redundantly”show the same lesion twice. It has been found that it is more desirableto provide “redundant” showings of the same mass lesion in two adjacentthick-slice images than to fail to display the “full effect” of a masslesion straddling the borders of those slices. Also, because the displayof the ultrasound thick-slices is only used in an adjunctive diagnosticsense, and not in an absolute diagnostic sense, there are few if anypractical implications for overall system specificity.

FIG. 14 illustrates steps for interactive display of adjunctiveultrasound mammography information according to a preferred embodiment.The steps of FIG. 14 are further illustrated in the adjunct displays ofFIGS. 15-17. At step 1402, an array of thick-slice images is displayedadjacent to a raw ultrasound slice and an orthogonal thin-slice image.With respect to FIG. 15, the array of thick-slice images 1502 isdisplayed next to planar ultrasound images 1504 including a rawultrasound image 1506 and an orthogonal thin-slice image 1508. At step1404, an active member of the thick-slice array is selected, such as byreceiving a simple mouse point-and-click from the user, and is thenhighlighted. With respect to FIG. 15, thick-slice array 1502 comprisesan active member 1510 having a highlighted marking 1519 shown thereon.First and second range markers 1513 and 1514, respectively, are providedon the raw ultrasound image 1506 and the orthogonal thin-slice image1508, respectively, marking the upper and lower border of thethick-slice region of the active slice 1510 at a horizontal positioncorresponding to a cursor marker 1512 thereon.

Superimposed on active member 1510 are first and second plane indicators1517 and 1518, respectively. The first plane indicator 1517 correspondsto a first plane through the breast volume as displayed by the rawultrasound image 1506, and is visually related thereto by means of acolor such as green that matches colored lines at the edges of the rawultrasound image 1506. Preferably, the colored lines at the edges of rawultrasound image 1506 extend from the posterior edge thereof across tothe current position of the range marker 1513, which makes it easier tokeep track of the first range marker 1513 as the cursor 1512 is moved bythe user. The plane indicator 1517 is likewise heavily drawn from theposterior edge to the cursor position and then lightly drawn over to theanterior edge to assist tracking. According to an alternative preferredembodiment, the first range marker 1513 can remain fixed at the centerof the frame, while the rest of the raw ultrasound slice dynamicallyshifts around it according to user cursor movements. Similardescriptions apply to second plane indicator 1518, orthogonal thin-sliceimage 1508, and second range marker 1514.

At step 1408 the position of cursor 1512 on active member 1510 isprojected onto other members of the thick-slice image array. Shown inFIG. to 15 with respect to a representative second member image 1515 isa projected cursor 1516. Also shown in the second member image 1515 areprojected first and second plane indicators 1517 a and 1518 a.

At step 1410 the user moves the cursor position on the activethick-slice image. At step 1412 in a real-time “cine-like” display isprovided wherein the planar images 1504 are refreshed in real-time tokeep up with the current position of plane indicators 1517 and 1518,which follow the cursor position. At step 1414, real-time updating ofthe cursor projections and the plane indicator projections is provided.According to a preferred embodiment, shifts in the position of cursor1512 and corresponding shifts in the plane indicators 1517 and 1518 aremirrored in the other members of the thick-slice array, as illustratedin FIG. 16. FIG. 16 also shows how range markers 1513 and 1514 areshifted to new positions 1613 x and 1614 x, respectively, during thecursor shift, as well as the manner in which the colored edges of theplanar images shift with cursor position.

Also shown in FIGS. 15 and 16 is a patient ID 1520 and athree-dimensional icon 1522. The three-dimensional icon 1522 comprises aframe structure 1524 similar to the frame structure 510 of FIG. 5, andtwo iconic plane indicators 1526 and 1528 thereon corresponding to planeindicators 1517 and 1518, respectively. As indicated in FIG. 16, theiconic plane indicators 1526 and 1528 also move dynamically with thecursor position. In one preferred embodiment, the look and feel of thecursor movement is analogous to a computer-aided drawing display (e.g.,Microsoft Visio) in which the cursor can, at the user's option, “stick”to horizontal and vertical “grid lines”. This allows for one of theplanar ultrasound images 1506 or 1508 to remain perfectly static if thecursor is moved along the horizontal or the vertical directions. Asdescribed supra, in an alternative preferred embodiment the rangemarkers 1513 and 1514 can remain fixed at the center of the planarultrasound frame, while the images move around them, it being found thatmany users can focus better on the tissue near the cursor position usingthis method.

In the display of FIGS. 15-16, soft buttons analogous to those of FIGS.4-11 are omitted in favor of a mechanical keypad described infra,although the scope of the preferred embodiments is by no means solimited. At step 1416, the displayed thick-slice images are inverted(see FIG. 8, supra) when the user presses an “invert” button on thekeypad. Preferably, when an inverted convention is used, the backgroundis segmented out and returned to dark. At step 1418 an enlarged view ofthe active thick-slice image is displayed if the user clicks or taps ona selected cursor position or enters a “one slab” keypad entry.

FIG. 17 illustrates an enlarged view 1702 of the active thick-sliceimage member 1510 of FIGS. 15-16 that appears at step 1418. The cursorposition 1704 corresponds to the same breast location as the selectedcursor position 1512 of FIG. 15. At step 1420, the user is permitted (oneither the one-slab or all-slab displays) to manipulate the orientationand positions of plane indicators 1706/1708 or 1517/1518 in a mannersimilar to the way a computer draftsperson manipulates lines in agraphical illustration, while in real-time the planar ultrasound images1504 remain updated to reflect corresponding planes through the breastvolume.

FIG. 18 illustrates an exterior view of an adjunctive ultrasoundmammography display unit 1802 in accordance with a preferred embodiment.Adjunctive ultrasound mammography display unit 1802 comprises an x-rayfilm conveyor and display unit 1804 with modifications made toaccommodate adjunctive ultrasound display. X-ray film conveyor anddisplay unit 1804 comprises a lower film conveyor 1806 and an upper filmconveyor 1812, the lower conveyor 1812 being turned off in this exampleand the lower conveyor 1806 displaying two x-ray mammogram images 1808and 1810. Conventional control buttons 1814 provided with mostcommercially available x-ray film conveyor and display units areprovided to control the x-ray mammogram film conveyors.

According to a preferred embodiment, adjunctive ultrasound mammographydisplay unit 1802 comprises a full-sized LCD display 1816 integratedinto a front table portion thereof as shown in FIG. 18. Preferably, LCDdisplay 1816 is large enough to display two full-scale thick-sliceultrasound images simultaneously, including a left image correspondingto the x-ray mammogram image 1808 and a right image corresponding tox-ray mammogram image 1810. A 17-inch diagonal LCD display has beenfound suitable to achieve this objective. In contrast to the preferredembodiment of FIG. 1, supra, LCD display 1816 is conveniently located toallow back-and-forth viewing between the thick-slice images and thex-ray images in an up-and-down manner, with minimal head motion. Asdiscussed previously, even subtle ergonomic issues can be of crucialimportance in the breast cancer screening process, because if the imageviewing apparatus leads to tired or physically fatigue radiologists,such fatigue can unfortunately transform into a potentially fatal misseddiagnosis or, alternatively, the inefficient process of an unnecessarypatient callback.

In distinction contrast to cathode ray tube (CRT) monitors of equivalentviewable size, the LCD display 1816 can be advantageously placed in themiddle of the table portion without interfering with the knees of theuser. Adjunctive ultrasound mammography display unit 1802 furthercomprises CRT monitors 1818 and 1820 directly to the left and right ofthe LCD display 1816, respectively. Preferably, the CRTs have a 7-inchdiagonal screen. It has been found more desirable to present the aboveplanar ultrasound images (e.g., raw ultrasound slices and orthogonalthin-slice views) on smaller CRT displays on the side of a larger LCDthick-slice display. The smaller CRT display allows for a substantiallyincreased brightness range and also represents a more familiar opticalcharacteristic to users familiar with traditional ultrasound displays.Among these characteristics is a slight low-pass filtering effect due tothe finite width of the cathode-ray beam, in distinction to apixellating effect that can be observed on LCD monitors. Because the CRTmonitors 1818 and 1820 are set back and to the side of the LCD monitor1816, they do not interfere with the knees of the user. Adjunctiveultrasound mammography display unit 1802 further comprises a mouse 1822and a keypad 1824 for controlling the adjunctive ultrasound mammographydisplays.

FIG. 19 illustrates the keypad 1822 of FIG. 18 in more detail. Thekeypad 1822 provides functionality similar to the soft buttons describedsupra with respect to FIG. 11, with some additional capabilitiesrelevant to the preferred embodiments of FIGS. 15-17. An “invert” key1902 causes the display of all thick-slice images to be inverted, asdescribed supra with respect to FIG. 14. Also, a “prev slab” key 1904and “next slab” key 1906 are provided as an additional method forallowing the user to change the active member of the thick-slice arraybeing displayed.

FIG. 20 illustrates an x-ray/adjunctive ultrasound mammography displayaccording to a preferred embodiment as implemented on the apparatus ofFIG. 18. An example is shown for the CC view, it being understood thatan equivalent display is also provided for the MLO view. The MLO x-rayviews may be shown, for example, on the upper conveyor 1812 of FIG. 18,while a toggle button allows the corresponding adjunctive ultrasoundimages for the MLO view to be displayed on monitors 1816, 1818, and1820. Shown in FIG. 20 is an RCC x-ray mammogram film 2002 and an LCCx-ray mammogram film 2004. On the left hand side of LCD display 1816 isa thick-slice array 2006 for the CC view of the right breast, and on theright hand side of LCD display 1816 a thick-slice array 2008 for the CCview of the left breast. CRT monitor 1818 displays a raw ultrasoundimage 2010 and orthogonal thin-slice view 2012 corresponding to acurrent active cursor position on the RCC thick-slice display 2006. CRTmonitor 1820 shows equivalent planar ultrasound images 2014 and 2016 forthe LCC thick-slice display 2008.

According to an alternative preferred embodiment, only a single CRTdisplay is provided to one side of the central LCD display, the centralLCD display showing thick-slice images for only a single breast, and theCRT display showing the corresponding planar ultrasound images. Thisalternative preferred embodiment can be advantageous for cost and spaceconsiderations, as well as in recognition of the fact that the user willgenerally only be closely analyzing a single breast of a time.

According to another alternative preferred embodiment, one side of FIG.20 displays CC views (CC x-ray, CC thick-slice images, and CC planarultrasound images) for a single breast, while the other side of FIG. 20displays MLO views (MLO x-ray, MLO thick-slice images, and MLO planarultrasound images) for that breast. This preferred embodiment can beadvantageous insofar as all standard information for the breast is nowprovided in a single set of images being concurrently displayed, ratherthan requiring the user to physically instantiate switches between CCand MLO views of the same breast during analysis.

A variety of other configurations, each having its particularadvantages, is also within the scope of the preferred embodiments. Byway of example, the thick-slice images may instead be provided on paperor on film, while the electronic display of thin-slice images are anoptional addition to the unit. Where the thick-slice images are providedon film, a technician can simply load the thick-slice image films intothe conveyor unit adjacent to or above the standard x-ray films.

According to another preferred embodiment relating generally to themethod of FIG. 14, the user may instantiate a cine display for the rawultrasound image 1506, the orthogonal thin-slice image 1508, or acombination of both along a predetermined cursor trajectory. By way ofexample, responsive the user pressing an additional button “lat-medcine” (not shown) on keypad 1822, a cine display of the raw ultrasoundslice 1506 is displayed with the cursor 1512/1704 moving automaticallyin a vertical direction on thick-slice image 1510/1702 at its currenthorizontal position. The orthogonal thin-slice image 1508 will remainstatic. The user can control the cine display using additional cinecontrol buttons (not shown) on keypad 1822. Likewise, responsive theuser pressing an additional button “post-ant cine” (not shown) on keypad1822, a cine display of the orthogonal thin-slice image 1508 isdisplayed with the cursor 1512/1704 moving automatically in a horizontaldirection on thick-slice image 1510/1702 at its current verticalposition, the raw ultrasound image 1506 remaining static. It ispreferable that this preferred embodiment be used in conjunction withthe particular above-described preferred embodiment in which the rangemarkers 1513 and 1514 are fixed at the center of the frame.

In a slightly more complicated preferred embodiment, controls areprovided for which the user can select a custom cine trajectory, whereinboth of the planar ultrasound images 1506 and 1508 will change as thecursor 1512/1704 automatically follows that trajectory. The user canselect the custom cine trajectory, for example, by manipulating theplane indicators 1517/1518 in a Visio-like manner as describedpreviously. The user can then activate the cine action by pressing the“lat-med” cine button to make the cursor 1512/1704 move along the planeindicator 1518, or by pressing the “lat-med” cine button to make thecursor 1512/1704 move along the plane indicator 1517.

According to another preferred embodiment, CAD markers identifyingsuspicious lesions are superimposed on the thick-slice images 1502 todirect the attention of the user to those particular locations.Preferably, the CAD markers are generated from the volumetric ultrasoundscan data used to generate the displayed thick-slice images 1502,although the scope of the preferred embodiments is not so limited.According to a preferred embodiment, the CAD markers are displayedresponsive to the user pressing a “show CAD marker” button (not shown)on keypad 1822, and are superimposed on the currently-showingthick-slice display, i.e. the thick-slice array 1502 or the enlargedthick-slice image 1702. The CAD markers may be color-coded, size-coded,shape-coded, mode-coded (blinking, flashing, etc.), etc., and/oraccompanied by nearby alphanumeric tags or text messages to convenientlyportray higher-suspiciousness lesions versus lower-suspiciousnesslesions. Preferably, where the user presses the “show CAD marker” buttonas the thick-slice array 1502 is displayed, the active thick-slice imageis automatically selected to be that member containing themost-suspicious lesion, the cursor 1512 is automatically relocated tothe location of that lesion, and the planar ultrasound images 1506 and1508 thereby automatically show the location of that lesion in thecenter of the planar image displays and mark that location with therange markers 1513-1514 and an optional additional indicator.

According to another preferred embodiment for use in conjunction witheach of the above embodiments is a reverse-locating capability from afirst location of interest on a planar ultrasound image to acorresponding location on the proper corresponding thick-slice image.During review of the planar ultrasound images 1506 and 1508 as the useris moving the cursor 1512/1704 around the active thick-slice memberimage 1510, the user may see an interesting location appear somewhere onone of the planar ultrasound images 1506 or 1508. When this occurs, theuser can right-click the mouse or press a “reverse locator” button (notshown) on keypad 1822. Responsive thereto, the cursor 1512/1704transforms into a reverse locator pointer, such as by turning into abright-yellow arrow. The user then moves this arrow over to the firstlocation of interest in the planar ultrasound image 1506 or 1508 andthen left-clicks or presses the “reverse locator” button again.Responsive to the selection of the first location of interest, (i) theyellow arrow disappears, (ii) the thick-slice image corresponding to thefirst location of interest becomes the active thick-slice member image,and (iii) the cursor 1512/1704 re-appears at a location corresponding tothat first location of interest. The reverse-locating capability isusable regardless of whether the current display mode is of thethick-slice array 1502 or the enlarged thick-slice view 1702. In theevent that the current display mode is the enlarged thick-slice view1702, the currently-displayed thick-slice image is replaced, ifnecessary, by a different thick-slice image corresponding to the depthof the first location of interest.

An adjunctive ultrasound system according to the preferred embodimentsdoes not supplant existing x-ray mammogram screening methods. Indeed,reference to the adjunctive ultrasound data is optional depending on thecontents of the x-ray mammogram image, and for many patients it may noteven be used at all. Rather, the adjunctive ultrasound system is thereto assist the radiologist in performing their pre-existing professionalduties with respect to “difficult” or “marginal” mammograms. As such, amedical establishment faces little risk of failure in acquiring anadjunctive ultrasound system according to the preferred embodiments. Ina worst-case scenario, the adjunctive ultrasound system would be metwith indifference by the entrenched “x-ray-only” believers, because itwould not disturb their pre-existing routines. However, the adjunctiveultrasound system will be there standing by to assist in the “difficult”cases, and it is expected that even the “x-ray-only” believers willeventually find the system useful and will increasingly rely on it toincrease their sensitivity and specificity performance.

Also within the scope of the preferred embodiments is a computer programproduct for instructing one or more processors to carry out one or moreof the methods of the preferred embodiments, such computer programproduct being amenable to ready implementation by a person skilled inthe art in view of the present disclosure. In one preferred embodiment,the computer program product is executed primarily by the ultrasoundserver 106 of FIG. 1, with the other system devices of FIG. 1 performingsimple input/output, display, and storage functions. In other preferredembodiments, the computer program product is distributed across thedifferent systems of FIG. 1, with different algorithmic portions beingcarried out by different systems or subsystems. Ultrasound server 106comprises a computer system that includes a cabinet, a display monitor,a keyboard, and a mouse, the mouse having one or more buttons forinteracting with a graphical user interface. The cabinet typicallyhouses a CD-ROM, zip, and/or floppy disc drive, system memory and a harddrive which can be utilized to store and retrieve software programsincorporating computer code that implements the preferred embodiments,data for use with the invention, and the like. An external hard drive isalso shown in FIG. 1. Although CD-ROM, zip, and floppy discs representcommon computer readable storage mediums, other computer readablestorage media including tape, flash memory, system memory, and harddrives can be used. Additionally, a data signal embodied in a carrierwave, such as in a network including the Internet or an intranet, canform the computer readable storage medium.

Whereas many alterations and modifications of the present invention willno doubt become apparent to a person of ordinary skill in the art afterhaving read the foregoing description, it is to be understood that theparticular embodiments shown and described by way of illustration are inno way intended to be considered limiting. By way of example, althoughdescribed supra in terms of adjunctive ultrasound screening, in view ofthe present disclosure one skilled in the art would readily be able toapply the thick-slice display apparatus of the preferred embodiments inthe context of computerized tomography (CT) and/or magnetic resonanceimaging (MRI) environments. In each case, individual image slicesgenerated from CT scans or MRI scans of the breast are compounded so asto form thick-slice images of slab-like portions of the breast alongplanes parallel to a standardized x-ray mammogram view plane, and thethick-slice images are displayed to the radiologist in close proximityto an x-ray mammogram of the breast to assist in interpreting that x-raymammogram. Preferably, a single composite view of the whole breast isshown together with the thick-slice image views, these views havingtheir gray-scale polarities toggled and/or remapped such that they arereminiscent of x-ray mammogram views taken from the standardizeddirection. By way of further example, the preferred embodimentsdescribed supra may also be used with different ultrasound modalitiesother than B-mode scans, including power or color Doppler modalities,and may also be used in conjunction with vibrational Doppler imaging(VDI) modalities. Therefore, reference to the details of the preferredembodiments are not intended to limit their scope, which is limited onlyby the scope of the claims set forth below.

1. A method for acquisition and interactive display of adjunctiveultrasound mammography information in a breast cancer screeningenvironment, comprising: compressing a breast of a patient along acompression direction by exerting a compressive force; acquiring arelatively high number of individual ultrasound slice imagesrepresentative of a sonographic property of substantially an entirevolume of the breast by scanning the breast in an automated scanningprocess with a linear array transducer, the individual ultrasound slicesconforming to planes that are generally along the compression directionand being acquired at a respective relatively high number of scanpositions spaced apart by less than about 0.75 mm; computing from saidrelatively high number of individual ultrasound slice images a pluralityof two-dimensional thick-slice images representative of the sonographicproperty of the breast in respective relatively thick slices thereofgreater than about 2 mm and less than about 20 mm in thickness, saidrespective relatively thick slices conforming to planes that aregenerally transverse to the compression direction, wherein saidplurality of thick-slice ultrasound images is sufficient in number, andsaid respective relatively thick slices are sufficient in thickness,such that said respective relatively thick slices collectively occupysubstantially the entire breast volume; electronically displaying to auser a thick-slice image array comprising all of said thick-sliceultrasound images, wherein all of said thick-slice ultrasound images aredisplayed in a simultaneous, side-by-side manner; receiving a firstcommand from the user that selects a first member of said thick-sliceultrasound image array and a first location thereon; responsive to saidfirst command, electronically displaying an enlarged thick-slice imageview of said selected first member; and causing to be displayed on saidenlarged thick-slice image view a viewable marker at a second locationspatially corresponding to a same location in the breast as said firstlocation on said selected first member.
 2. The method of claim 1,wherein said first command consists of single-movement input thatsimultaneously selects said first member and said first location, saidsingle-movement input being selected from the group consisting of: amouse point-and-click, a touchscreen stylus point-and-click, a mousepoint-and-double-click, and a touchscreen stylus point-and-double-click.3. The method of claim 1, said thick-slice image array and said enlargedthick-slice image view being displayed on a thick-slice display, furthercomprising: receiving cursor movement commands from the user; moving acursor position on said thick-slice display responsive to said cursormovement commands to a third location that is either on said enlargedthick-slice image view or on a member of said thick-slice image array;displaying adjacent to said thick-slice display and concurrentlytherewith a first planar ultrasound image representing the sonographicproperty of the breast along a first plane substantially perpendicularto the thick slice corresponding to the thick-slice image underlyingsaid third location; and updating in real time said first planarultrasound image according to said third location as said cursorposition is moved.
 4. The method of claim 3, farther comprising:displaying adjacent to said first planar ultrasound image andconcurrently therewith a second planar ultrasound image representing thesonographic property of the breast along a second plane different thansaid first plane, said second plane also being substantiallyperpendicular to the thick slice corresponding to the thick-slice imageunderlying said third location; and updating in real time said secondplanar ultrasound image according to said third location as said cursorposition is moved.
 5. The method of claim 4, wherein said second planeis orthogonal to said first plane.
 6. The method of claim 5, whereinsaid compression direction is at an angle that corresponds to a standardx-ray mammogram view.
 7. The method of claim 1, said thick-slice imagearray and said enlarged thick-slice image view being displayed on athick-slice display, further comprising: receiving an inversion commandfrom the user; responsive to said inversion command, displaying aninverted version of said thick-slice display having an ultrasoundbrightness convention substantially opposite an ultrasound brightnessconvention of said thick-slice display.
 8. The method of claim 1,wherein said number of thick-slice ultrasound images in said thick-sliceimage array is between about 4 and
 16. 9. The method of claim 1, furthercomprising, while all of said thick-slice ultrasound images of saidthick-slice image array are displayed in said simultaneous, side-by-sidemanner: receiving a second command from the user that selects a secondmember of said thick-slice ultrasound image array and a second locationthereon; displaying on each member of said thick-slice image array otherthan said second member a projected cursor at a location thereon thatcorresponds in lateral position to said second location on said secondmember; computing, based on said second location and from saidrelatively high number of individual ultrasound slice images, first andsecond planar images corresponding to first and second planes in thebreast generally perpendicular to each other and generally along saidcompression direction; and displaying first and second planar ultrasoundimages adjacent to and concurrently with said thick-slice image array.10. The method of claim 9, further comprising, while all of saidthick-slice ultrasound images of said thick-slice image array aredisplayed in said simultaneous, side-by-side manner: receiving cursormovement commands from the user operative to move said second locationprogressively across said second member; and dynamically recomputing andredisplaying said projected cursors and said first and second planarimages in real time as said cursor movement commands are received.